Zoom control device, zoom control method, and program

ABSTRACT

A zoom control device includes: a zoom magnification ratio change speed setting unit that sets a main image zoom magnification ratio change speed and a monitoring image zoom magnification ratio change speed according to a zoom operation by a user; and a zoom control unit that conducts a zoom control on a main image so that a zoom magnification ratio changes according to the main image zoom magnification ratio change speed, and conducts a zoom control on a monitoring image so that the zoom magnification ratio changes according to the monitoring image zoom magnification ratio change speed. The zoom magnification ratio change speed setting unit is configured to set the main image zoom magnification ratio change speed by smoothing the monitoring image zoom magnification ratio change speed.

TECHNICAL FIELD

The present disclosure relates to a zoom control device, a zoom controlmethod, and a program.

BACKGROUND ART

In a camera system, a user may shoot a video while executing a zoomfunction by operating a zoom key (see Patent Literature 1, for example).

CITATION LIST Patent Literature

-   -   Patent Literature 1: JP 2004-252370A

DISCLOSURE OF INVENTION Technical Problem

If a user who is not used to zoom operations shoots a video, in somecases a scene may be shot in which the subject is zoomed in (enlarged)or zoomed out (reduced) at sudden zoom speeds due to inappropriate zoomoperations. Since such scenes risk making a viewer who views the videouncomfortable, improvement is desired.

Consequently, one objective of the present disclosure is to provide azoom control device, a zoom control method, and a program devised inlight of the above point.

Solution to Problem

To solve the problem, the present disclosure is, for example, a zoomcontrol device, including: a zoom magnification ratio change speedsetting unit that sets a main image zoom magnification ratio changespeed and a monitoring image zoom magnification ratio change speedaccording to a zoom operation by a user; and a zoom control unit thatconducts a zoom control on a main image so that a zoom magnificationratio changes according to the main image zoom magnification ratiochange speed, and conducts a zoom control on a monitoring image so thatthe zoom magnification ratio changes according to the monitoring imagezoom magnification ratio change speed. The zoom magnification ratiochange speed setting unit is configured to set the main image zoommagnification ratio change speed by smoothing the monitoring image zoommagnification ratio change speed.

The present disclosure is, for example, a zoom control method,including: setting, by a zoom magnification ratio change speed settingunit, a main image zoom magnification ratio change speed and amonitoring image zoom magnification ratio change speed according to azoom operation by a user; and conducting, by a zoom control unit, a zoomcontrol on a main image so that a zoom magnification ratio changesaccording to the main image zoom magnification ratio change speed, andconducting a zoom control on a monitoring image so that the zoommagnification ratio changes according to the monitoring image zoommagnification ratio change speed. The zoom magnification ratio changespeed setting unit sets the main image zoom magnification ratio changespeed by smoothing the monitoring image zoom magnification ratio changespeed.

The present disclosure is, for example, a program causing a computer toexecute a zoom control method including: setting, by a zoommagnification ratio change speed setting unit, a main image zoommagnification ratio change speed and a monitoring image zoommagnification ratio change speed according to a zoom operation by auser; and conducting, by a zoom control unit, a zoom control on a mainimage so that a zoom magnification ratio changes according to the mainimage zoom magnification ratio change speed, and conducting a zoomcontrol on a monitoring image so that the zoom magnification ratiochanges according to the monitoring image zoom magnification ratiochange speed. The zoom magnification ratio change speed setting unitsets the main image zoom magnification ratio change speed by smoothingthe monitoring image zoom magnification ratio change speed.

The present disclosure is, for example, a zoom control device,including: a monitoring image zoom magnification ratio control unit thatcontrols an operation of changing a monitoring image zoom magnificationratio according to a monitoring image zoom magnification ratio changespeed based on a zoom operation by a user; and a main image zoommagnification ratio control unit that controls an operation of changinga main image zoom magnification ratio according to a main image zoommagnification ratio change speed obtained by conducting a smoothingprocess on the monitoring image zoom magnification ratio change speed.

Advantageous Effects of Invention

According to at least one embodiment, it is possible to obtain an imagein which appropriate zoom control has been performed. However, theadvantageous effect described herein is not necessarily limited, and mayalso be any of the advantageous effects described in this disclosure.Additionally, the content of the present disclosure is not to beinterpreted as being limited by the advantageous effects given as anexample.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for explaining an overview of an embodiment of thepresent disclosure.

FIG. 2 is a block diagram illustrating an example of a configuration ofan imaging device according to a first embodiment of the presentdisclosure.

FIG. 3 is a diagram for explaining an example of a configuration of azoom control unit.

FIG. 4 is a diagram for explaining an example of a process by asmoothing processing unit.

FIG. 5 is a diagram for explaining an example of the impulse response ofa low-pass filter.

FIG. 6 is a flowchart illustrating an example of the flow of a processaccording to a first embodiment of the present disclosure.

FIG. 7 is a diagram for explaining an example of a process of updating atarget zoom magnification ratio.

FIG. 8 is a diagram illustrating an example of a degree of smoothingaccording to imaging conditions.

FIG. 9 is a flowchart illustrating an example of the flow of a processaccording to a second embodiment of the present disclosure.

FIGS. 10A to 10E are diagrams for explaining a synchronization processaccording to a third embodiment of the present disclosure.

FIG. 11 is a flowchart illustrating an example of the flow of a processaccording to a third embodiment of the present disclosure.

FIG. 12 is a diagram for explaining an example of the relationshipbetween an in-image position and an electronic zoom magnification ratioin a case of conducting an enlargement process by zooming.

FIG. 13 is a diagram for explaining an example of the relationshipbetween an in-image position and an electronic zoom magnification ratioin a case of conducting an enlargement process by zooming.

FIG. 14 is a diagram illustrating an example of monitoring images, mainimages, and the like according to zoom operations, according to a fourthembodiment.

FIG. 15 is a diagram for explaining a modification.

FIG. 16 is a diagram for explaining a modification.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, multiple embodiments of the present disclosure will bedescribed with reference to the drawings. Note that the description willproceed in the following order.

-   <1. First Embodiment>-   <2. Second Embodiment>-   <3. Third Embodiment>-   <4. Fourth Embodiment>-   <5. Modifications>

The embodiments and the like described hereinafter are specificfavorable examples of the present disclosure, but the content of thepresent disclosure is not limited to these embodiments and the like.

OVERVIEW OF PRESENT DISCLOSURE

First, an overview of the present disclosure will be described withreference to FIG. 1. In FIG. 1, the horizontal axis represents the timeaxis, while the vertical axis represents the zoom magnification ratio.Also, in FIG. 1, examples of images corresponding to zoom magnificationratios are illustrated.

For example, at the timing of time t0, the user uses an imaging deviceto start shooting a video. The shot video may be recorded in the imagingdevice, or may be transmitted to external equipment without beingrecorded. For example, by transmitting the shot video without recording,the conditions of the subject being shot may be checked at a remotelocation or the like (also known as live streaming). Additionally, aprocess for displaying an already-shot video may be considered playback.This playback is not limited to playing back videos recorded in theimaging device, and may also be conducted outside the imaging device.Additionally, a video recorded in the imaging device may be read outfrom a recording medium, but the display thereof may be presented onexternal equipment. The initial image, which is the image M1, includesmultiple houses and a mountain, for example. At the timing of time t1,the user operates a zoom key. The change in the zoom magnification ratioin response to the zoom key operation is indicated by the dashed line.According to the zoom key operation, the area near the mountain iszoomed in.

At this point, if the user's zoom operation is not appropriate, such asif the operation magnitude on the zoom key is too large, for example,the area near the mountain is enlarged too much (image M2 at timing t2).To achieve the intended zooming, the user finely adjusts the zoom key tozoom out (the operation from timing t2 to timing t3), and obtains theimage M3 in which the mountain is enlarged to a suitable degree. Whenplaying back a video recorded in this way, since the area near themountain is zoomed in suddenly and then zoomed out, the video includesan unnecessary change of zoom magnification ratio, and may make viewersuncomfortable in some cases. Also, zooming at an excessive change speedof the zoom magnification ratio (the excessive slope between t1 and t2in FIG. 2) and sudden accelerations or decelerations in the zoommagnification ratio (the discontinuities in the slope at t1 and t2) mayalso make viewers uncomfortable in some cases.

If the user performs appropriate zoom operations, the above problems donot occur. However, in recent years, as imaging devices become morecompact, the operating units that accept zoom operations likewise arebecoming more compact. For this reason, it is becoming difficult for auser to perform appropriate zoom operations reliably and capture asubject at the intended size.

On the other hand, as a potential technology (not technology of therelated art), it is conceivable to avoid the above problems by uniformlyslowing down the change speed of the zoom magnification ratio (zoommagnification ratio change speed). However, there is a risk of givingthe user the impression of lowered response to zoom operations.Additionally, since the time until a zoom change finishes becomes longeraccording to the slowing down of the zoom magnification ratio changespeed, the time required for zoom operations increases, which raises therisk of missing the timing for other shooting operations. Consequently,uniformly slowing down the zoom magnification ratio change speed is notrealistic.

Accordingly, as an example in an embodiment of the present disclosure,as indicated by the solid line in FIG. 1, for a main image, the zoommagnification ratio change speed is slowed down and set to accelerateand decelerate smoothly, and an image in which the zoom has beencontrolled by the zoom magnification ratio indicated by the solid lineis obtained. This image may be recorded as appropriate. On the otherhand, for the image displayed on a device such as viewfinder or an LCDmonitor during shooting, there is displayed an image in which the zoomhas been controlled by a zoom magnification ratio reflecting the user'szoom operations (a zoom magnification ratio which is the same as, orwhich approximates, the zoom magnification ratio indicated by the dashedline). Hereinafter, multiple embodiments of the present disclosure willbe described in detail.

Note that the zoom magnification ratio change speed in the followingdescription is prescribed the change or the proportional change of thezoom magnification ratio, but is not limited thereto. For example, thezoom magnification ratio change speed may also be prescribed by theproportional change of the field of view, the proportional change of thefocus distance, the proportional change of the position of the zoomlens, the amount of movement per unit time of the zoom lens, or thelike.

For example, the dashed line in FIG. 1 expresses the state of change inthe zoom magnification ratio obtained in response to zoom operations(hereinafter designated the target zoom magnification ratio whereappropriate). The solid line in FIG. 1 expresses the state of change ina zoom magnification ratio controlled so that the input target zoommagnification ratio changes smoothly (hereinafter designated the mainimage zoom magnification ratio where appropriate). Additionally, themagnitude of the change in the slope of the dashed line in FIG. 1 isdesignated the target zoom magnification ratio change speed whereappropriate, while the magnitude of the change in the slope of the solidline is designated the main image zoom magnification ratio change speed.Note that the zoom position refers to the lens position for shooting ata certain zoom magnification ratio. Zoom control is conducted so thatthe image presented to the user during shooting (monitoring image) is animage according to the target zoom magnification ratio. To conduct thiszoom control, a monitoring image zoom magnification ratio is set. In thepresent disclosure, as an example, the completion of a main image zoommagnification ratio change operation is set to be later than thecompletion of a monitoring image zoom magnification ratio changeoperation.

1. First Embodiment [Configuration of Imaging Device]

An embodiment of the present disclosure is an example of applying a zoomcontrol device to an imaging device that includes a video imagingfunction. Specific examples of the imaging device may include devicessuch as a digital still camera, a digital video camera, a smartphone, amobile phone, a remotely controllable robot, and a remotely controllablesecurity camera. However, the content of the present disclosure is notlimited to an example in which the zoom control device is built into theimaging device.

FIG. 2 illustrates an example of the configuration of the imagingdevice. Note that the solid arrows in FIG. 2 indicate an example of theflow of image data, while the dashed arrows indicate an example of theflow of control signals and commands.

The imaging device 1 is equipped with a lens block 101, an image sensor102, a digital signal processing unit 103, a recording device 104, adisplay device 105, a control unit 106, an operating unit 107, and alens driving unit 108. The digital signal processing unit 103 isequipped with image processing function blocks, such as a preprocessingunit 111, a synchronization (demosaicing) unit 112, a YC generation unit113, a resolution conversion unit 114, a codec unit 115, and anelectronic zoom processing unit 116, for example. The control unit 106includes, as one function block, a zoom control unit 120 correspondingto the zoom control device.

The lens block 101 is equipped with lenses such as a cover lens, a zoomlens, and a focus lens, as well as a diaphragm mechanism. Light from asubject is condensed onto the image sensor 102 through the lens block101. The lens block 101 may also be an interchangeable lens that may beinterchanged on the imaging device 1.

The image sensor 102 includes an image sensor, such as thecharge-coupled device (CCD) type or the complementarymetal-oxide-semiconductor (CMOS) type, for example. In the image sensor102, processes such as a correlated double sampling (CDS), an automaticgain control (AGC) process, and an analog/digital (A/D) conversionprocess are performed on an electrical signal obtained by photoelectricconversion in the image sensor. An imaging signal converted into digitaldata is output from the image sensor 102 to the downstream digitalsignal processing unit 103.

The digital signal processing unit 103 is configured as an imageprocessing processor by a large-scale integrated circuit (LSI circuit),a digital signal processor (DSP), or the like. The digital signalprocessing unit 103 executes various types of signal processing ondigital image data supplied from the image sensor 102, according tocontrol by the control unit 106, for example.

The preprocessing unit 111 performs processes on the digital image datafrom the image sensor 102, such as a clamping process that clamps the R,G, and B black levels to a certain level, and an interpolation processbetween each of the R, G, B, color channels. The synchronization unit112 performs a color separation process to make the image data for eachpixel have color components for all of red (R), green (G), and blue (B).

The YC generation unit 113 generates (separates) a luminance (Y) signaland a chrominance (C) signal from R, G, and B image data. The resolutionconversion unit 114 executes a resolution conversion process on theimage data that has been subjected to various types of signalprocessing. The image processing in the digital signal processing unit103 is not limited to the processing discussed above. The digital signalprocessing unit 103 may also conduct known image processing differentfrom the image processing given as an example herein.

The digital image data that has been subjected to image processing isinput into each of the codec unit 115 and the electronic zoom processingunit 116. In the first embodiment, the image input into the codec unit115 corresponds to the main image in which zoom control already has beenperformed by optical zoom. Meanwhile, the image input into theelectronic zoom processing unit 116 corresponds to the monitoring imagebefore zoom control is performed by optical zoom and before zoom controlis performed by electronic zoom.

The codec unit 115 conducts a coding process for recording ortransmission, for example, on the resolution-converted digital data. Thecodec unit 115 also conducts a decoding process on digital image datainput from the recording device 104.

The electronic zoom processing unit 116 conducts an electronic zoomprocess on the input monitoring image, according to control by thecontrol unit 106. For example, an electronic zoom process is conductedin which part of the monitoring image is cut out and enlarged byinterpolating pixels, or reduced. Note that the content of theelectronic zoom process is not limited to any particular content,insofar as the image is enlarged or reduced by signal processing.

The recording device 104 includes a driver that conducts a recordingprocess on a recording device. The driver may conduct not only arecording process, but also a playback process of reading out data froma recording device. Note that the recording device may be one that isbuilt into the imaging device 1, such as a hard disk, or one that isfreely removable from the imaging device 1, such as semiconductormemory. Furthermore, the recording device 104 may also be separate fromthe imaging device 1. Digital image data may be transmitted between theimaging device 1 and the recording device 104 in a wired or wirelessmanner.

The display device 105 includes a driver. The driver conducts a displaycontrol corresponding to the type of monitor, and causes the monitor todisplay an image. The image may be a still image or a moving image. Onthe monitor, a monitoring image or a playback image, and a menu screenfor configuring various modes and settings in the imaging device 1, aredisplayed. The monitor is made up of a liquid crystal display (LCD),organic electroluminescence (EL), or the like. The monitor of thedisplay device 105 may also be configured as a touch panel and fulfillsome of the functions of the operating unit 107. The monitor may also bedisplay equipment that is connected to the imaging device 1 through aninterface.

The control unit 106 is made up of a microprocessor, such as a centralprocessing unit (CPU), for example, and controls each component of theimaging device 1. For example, the control unit 106 controls the lensdriving unit 108 to thereby move the focus lens and the like to acertain position. Additionally, the control unit 106 supplies operationtimings to the image sensor 102. Furthermore, the control unit 106performs control according to an operating signal input from theoperating unit 107, and as a result, the imaging device 1 operatesaccording to user operations.

The control unit 106 includes the zoom control unit 120 as a functionblock. Note that in FIG. 2, the zoom control unit 120 is built into thecontrol unit 106, but the control unit 106 and the zoom control unit 120may also be configured by different CPUs or the like.

The zoom control unit 120 corresponds to a zoom magnification ratiochange speed setting unit and a zoom control unit. The zoom control unit120 sets the main image zoom magnification ratio and the monitoringimage zoom magnification ratio according to zoom operations by the user.The monitoring image zoom magnification ratio is set based on the targetzoom magnification ratio. Additionally, the zoom control unit 120conducts zoom control on the main image with the main image zoommagnification ratio, and conducts zoom control on the monitoring imagewith the monitoring image zoom magnification ratio. The zoom controlunit 120 holds the main image zoom magnification ratio change speed andthe target zoom magnification ratio change speed as internal parameters,and these are set so that a main image zoom magnification ratio changeoperation is completed later than a monitoring image zoom magnificationratio change operation.

From a different perspective, the zoom control unit 120 corresponds to amain image zoom magnification ratio control unit and a monitoring imagezoom magnification ratio control unit. The main image zoom magnificationratio control unit controls the optical zoom based on the main imagezoom magnification ratio. The monitoring image zoom magnification ratiocontrol unit, in addition to the optical zoom control by the main imagezoom magnification ratio control unit, also sets an electronic zoommagnification ratio (to be discussed later in detail) and controls theelectronic zoom.

The lens driving unit 108 is controlled by the zoom control unit 120,and the zoom lens is moved to a certain position according to thiscontrol. Consequently, optical zooming is performed on the main imageand the monitoring image. Additionally, the zoom control unit 120 sets acertain zoom magnification ratio for the electronic zoom processing unit116. The zoom magnification ratio of the electronic zoom is set based onthe difference between the zoom magnification ratio indicated by thedashed line and the zoom magnification ratio indicated by the solid lineillustrated in FIG. 1, for example. As a result of the electronic zoomprocessing unit 116 according to the set zoom magnification ratio,electronic zooming is performed on the monitoring image. In other words,the zoom control on the monitoring image according to the firstembodiment is a compound control made up of an optical zoom control, andan electronic zoom control on the image obtained according to theoptical zoom control. Note that the detailed configuration of the zoomcontrol unit 120 will be discussed later.

The operating unit 107 includes an input function of accepting useroperations, and sends an operating signal corresponding to an inputoperation to the control unit 106. The operating signal is a digitizedsignal, for example. The operating unit 107 is realized by componentssuch as various operating elements provided on the housing of theimaging device 1, and a touch panel formed in the display device 105,for example. The operating elements on the housing may be a playbackmenu launch button, an OK button, directional keys, a cancel button, azoom key, a slide key, a shutter button (release button), and the like.The operating unit 107 may also be a remote control device capable ofoperating the imaging device 1 remotely.

According to operations on the zoom key, electrically-powered zooming isperformed, for example. On the zoom key, slide operations in theup-and-down direction or the left-and-right direction are possible. Oneof the two directions is set to the telephoto (T) side, while the otheris set to the wide-angle (W) side. The user of the imaging device 1 isable to give an instruction to increase the zoom magnification ratio bysliding the zoom key to the telephoto side, and is able to give aninstruction to decrease the zoom magnification ratio by sliding the zoomkey to the wide-angle side, for example. Note that the zoom key is notlimited to a mechanical key. For example, instructions to increase ordecrease the zoom magnification ratio may also be given according totouch operations on the touch panel, or instructions to increase ordecrease the zoom magnification ratio may also be given by voice.

Examples of the zoom driving method include the fixed-step method andthe continuous method. The fixed-step method refers to a method in whichseveral zoom magnification ratios are fixed in advance, such as the fivefixed zoom magnification ratios of actual size (×1), ×1.4, ×2, ×3, ×4,and ×5 being available, for example, and the zoom lens is driven by thelens driving unit 108 in accordance with the fixed zoom magnificationratio.

The continuous method refers to a method in which the zoom lens isdriven by the lens driving unit 108 so that the zoom magnification ratiovaries continuously from a lower-limit value to an upper-limit value. Inthis case, the lower-limit value of the zoom magnification ratio ispre-associated with the lower-limit position on the movable range of thezoom key, while the upper-limit value of the zoom magnification ratio ispre-associated with the upper-limit position. Specifically, for example,the lower-limit value of the zoom magnification ratio is taken to beactual size (1×), while the upper-limit value of the zoom magnificationratio is taken to be 5×. In this case, for example, the respectivevalues obtained by dividing the range of the zoom magnification ratiofrom 1× to 5× into 100 equal divisions are associated with therespective positions obtained by dividing the movable range of the zoomkey into 100 equal divisions. In this way, when the continuous method isadopted, the respective values of the zoom magnification ratio aremapped into the movable range of the zoom key. In the descriptionhereinafter, the continuous method is described as being the zoomdriving method, but the zoom driving method is not limited thereto.

The zoom control unit 120 periodically monitors operations on the zoomkey (such as the operation direction and the operation strength oroperation magnitude), for example. The period does not necessarily matchthe frame rate, and may be set according to the specifications of thesystem. In the case of optical zoom, the zoom lens is driven at amovement speed according to the operation. For example, if the zoom keyis long-pressed to the telephoto side, the zoom lens is displaced at afast movement speed. For this reason, the image is displayed to beenlarged suddenly. Note that the boundary (upper limit) of the movementspeed of the zoom lens is different depending on the imaging device. Inthe case of electronic zoom, a process of enlarging the displayed imageaccording to the operation is conducted. For example, if the zoom key islong-pressed to the telephoto side, electronic zooming is conductedevery 10 frames. For this reason, the image is displayed to be enlargedsuddenly. Note that the above example is merely one example, and the wayin which to conduct a zoom process according to a zoom operation may bemodified as appropriate.

The lens driving unit 108 includes a driving mechanism such as astepping motor that drives the zoom lens and the like. The lens drivingunit 108 moves the zoom lens to a certain position in accordance withcontrol by the zoom control unit 120, for example. Note that the lensdriving unit 108 may also be configured to include a microprocessor.Additionally, the microprocessor of the lens driving unit 108 maycommunicate with the control unit 106, and zoom lens positioninformation and the like may be supplied from the lens driving unit 108to the control unit 106, for example. The control unit 106 computes thedifference from the displaced position of the zoom lens based on thezoom lens position information, and controls the lens driving unit 108to displace the zoom lens in a certain direction and with a suitableamount of movement, for example.

The above thus describes an example of the configuration of the imagingdevice 1. Obviously, components different from the components given asan example may be added to the imaging device 1, and some of the abovecomponents may also be separate from the imaging device 1.

For example, an external input/output terminal may also be provided onthe imaging device 1. The imaging device 1 is connected to an externaldisplay device, such as a television set or a personal computer, forexample, through the external output terminal. A video or the likerecorded by the imaging device 1 may be displayed on the externaldisplay device.

The imaging device 1 may also be provided with a communication unit ableto connect to a network such as the Internet. Additionally, the imagingdevice 1 may also be provided with a communication unit that conductsshort-range wireless communication with external equipment. The imagingdevice 1 may also be provided with a sensor such as a gyro sensor, andmay be configured to be able to identify the directions in which theimaging device 1 moves (movement in the pan direction and movement inthe tilt direction) from information obtained by the sensor.

[Configuration of Zoom Control Device]

FIG. 3 is a diagram for explaining an example of a configuration of thezoom control unit 120, which is an example of a zoom control device. Thezoom control unit 120 is equipped with a target zoom management unit125, a smoothing unit 126, and a difference detection unit 127, forexample.

An operating signal based on a zoom operation performed using theoperating unit 107 is input into the target zoom management unit 125.The target zoom management unit 125 computes a zoom magnification ratiochange speed corresponding to the operating signal, and holds this zoommagnification ratio change speed as the target zoom magnification ratiochange speed. The target zoom management unit 125 supplies the targetzoom magnification ratio change speed to the smoothing unit 126 and thedifference detection unit 127.

The smoothing unit 126 is made up of a low-pass filter (LPF),specifically a finite impulse response (FIR) filter, for example. In thecase in which the smoothing unit 126 is made up of an FIR filter, thecoefficients of the FIR filter correspond to an example of parametersfor the smoothing process. In the first embodiment, the coefficients ofthe FIR filter are fixed.

The smoothing unit 126 obtains a smoothed target zoom magnificationratio change speed by smoothing the target zoom magnification ratiochange speed, and obtains the main image zoom magnification ratio byadding, to the main image zoom magnification ratio, a value obtained bymultiplying the smoothed zoom magnification ratio change speed by acontrol unit time. The smoothing unit 126 supplies the main image zoommagnification ratio to the difference detection unit 127. Additionally,the smoothing unit 126 conducts zoom control on the lens driving unit108 according to the main image zoom magnification ratio. According tothis zoom control, the lens driving unit 108 displaces the zoom lens toperform optical zooming.

The difference detection unit 127 obtains an electronic zoommagnification ratio based on the target zoom magnification ratio and themain image zoom magnification ratio. The difference detection unit 127computes the difference between the target zoom magnification ratio andthe smoothed target zoom magnification ratio, and decides the electroniczoom magnification ratio to compensate for this difference, for example.In addition, the monitoring image is always displayed at the target zoommagnification ratio, regardless of the state of the main image zoommagnification ratio. The difference detection unit 127 supplies themonitoring image zoom magnification ratio change speed to the electroniczoom processing unit 116, and conducts zoom control on the monitoringimage. According to this control, the electronic zoom processing unit116 executes electronic zooming on the monitoring image.

Since the main image zoom magnification ratio is set so that the zoommagnification ratio varies smoothly by the work of the smoothing unit126, the image obtained through the optical zooming zooms in or zoomsout on the subject smoothly. Consequently, even when the image is playedback, the subject is not zoomed in or zoomed out suddenly, and a senseof discomfort in a user viewing the image may be prevented. With respectto the target zoom magnification ratio according to zoom operations bythe user (for example, the dashed line illustrated in FIG. 1), theactual optical zooming becomes the main image zoom magnification ratio(for example, the solid line illustrated in FIG. 1). The electronic zoommagnification ratio is set to a value in a direction to compensate forthe difference between the two, or in other words, to cancel out thesmoothing. For this reason, an image reflecting zooming based on theactual zoom operations may be provided to the user while shooting.

[Operation of Smoothing Unit]

An example of the operation of the smoothing unit 126 will be describedspecifically. Note that in the following formulas, V represents speed,while Z represents the position of the zoom lens. Also, the values of Vand Z are expressed in a logarithmic representation. The smoothing unit126 performs the computation expressed by the following Math. 1, forexample.

$\begin{matrix}{V_{stabilized} = {{f\left( {Z_{target},Z_{capture}} \right)} \times {\max \left( {{{LPF}\left( V_{input} \right)},\frac{Z_{target} - Z_{capture}}{kT},ɛ} \right)}}} & \left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack\end{matrix}$

In Math. 1, Vstabilized means the stabilized speed, and represents themain image zoom magnification ratio change speed. f(Ztarget, Zcapture)represents a deceleration coefficient. LPF(Vinput) is the result ofapplying the LPF to the target zoom magnification ratio change speedaccording to the user's zoom operations, and smoothing the change ofspeed. ε is a constant of small value, used so that the zoom speed doesnot become too slow. kt is a constant for computing the zoom speed fromthe difference between the target zoom magnification ratio and the mainimage zoom magnification ratio. Ztarget is the target zoom magnificationratio corresponding to the user operations, and corresponds to the solidline in FIG. 1. Zcapture represents the current zoom lens position (orthe zoom magnification ratio obtained at that position).

FIG. 4 is a diagram for explaining an example of a decelerationcoefficient. In FIG. 4, the horizontal axis represents the absolutevalue of the difference between Ztarget and Zcapture(|Ztarget−Zcapture|), while the vertical axis represents an example ofthe deceleration coefficient. As the absolute value (|Ztarget−Zcapture|)becomes smaller, f(Ztarget, Zcapture) is set to be small. When theabsolute value (|Ztarget−Zcapture|) becomes a certain value or greater,f(Ztarget, Zcapture) is set to 1.

In other words, when the difference between the target zoommagnification ratio and the zoom magnification ratio obtained by thecurrent position of the zoom lens is small, f(Ztarget, Zcapture) becomessmall, and thus the zoom change speed decelerates, decelerating smoothlyto a stop at the target position. When the difference between the targetzoom magnification ratio and the zoom magnification ratio obtained bythe current position of the zoom lens is large, f(Ztarget, Zcapture)approaches 1, and thus the deceleration effect does not engage.

Note that when there are no zoom operations by the user, the value ofLPF(Vinput) becomes 0, and if there is no E, the value of Vstabilizedbecomes 0. In other words, the zoom lens stops. By setting E, the zoomlens may be prevented from stopping, and the zoom lens may be displacedto the position corresponding to the target zoom magnification ratio.

Note that a limit (Vlimit) may also be applied to Vstabilized asexpressed in Math. 2 below, so that the driving speed of the zoom lensdoes not exceed the capability of the lens driving unit 108, or so thatzooming is not conducted at an excessive speed leading to discomfortduring viewing.

$\begin{matrix}{V_{output} = \left\{ \begin{matrix}{V_{stabilized},} & {V_{stabilized} < V_{limit}} \\{V_{limit},} & {V_{stabilized} \geq V_{limit}}\end{matrix} \right.} & \left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack\end{matrix}$

If Vstabilized is less than Vlimit, Vstabilized is output as Voutput. IfVstabilized is equal to or greater than Vlimit, Vlimit is output asVoutput.

The smoothing unit 126 additionally computes the specific position ofthe zoom lens according to Math. 3 below.

Z _(update) =Z _(capture) +V _(output) ×T  [Math. 3]

In Math. 3, Zupdate represents the position of the zoom lens in the nextcontrol cycle (for example, the next frame), while Zcapture representsthe position of the zoom lens in the current control cycle (for example,the current frame). T represents the control interval, and representsthe frame rate (more specifically, 1/60), for example. The speed isconverted into a position by multiplying Voutput by T, and an updatevalue is obtained. By adding this update value to the current zoom lensposition, the position of the zoom lens in the next frame is computed,and optical zooming is conducted according to the new zoom magnificationratio. The smoothing unit 126 outputs Zupdate to the lens driving unit108, and the lens driving unit 108 controls the zoom position of thezoom lens to match the magnification ratio expressed by Zupdate.

FIG. 5 illustrates an example of the filter process LPF(Vinput) impulseresponse of the smoothing unit 126. As discussed earlier, the smoothingunit 126 is made up of a digital filter (FIR filter), for example. InFIG. 5, the dashed line schematically represents the impulse response inthe case of not applying the LPF. The output when the LPF is applied tothis impulse response is illustrated schematically by the solid line inFIG. 5. Note that the horizontal widths of the solid line in FIG. 5correspond to the control interval (T in Math. 2).

[Process Flow]

FIG. 6 is a flowchart illustrating an example of the flow of processesaccording to the first embodiment. Note that each process describedhereinafter is executed by the zoom control unit 120, unlessspecifically indicated otherwise. FIG. 6 illustrates processes incertain control interval units.

In step ST101, input via the zoom key in the operating unit 107 ismonitored, and the zoom operation state is acquired. The zoom controlunit 120 acquires the zoom operation state on a certain period, forexample. Subsequently, the process proceeds to step ST102.

In step ST102, the target zoom magnification ratio is updated accordingto the zoom operation. FIG. 7 is a diagram for explaining an example ofa target zoom magnification ratio update process, and is an example inwhich the process of step ST102 is realized by hardware. For example,the update value of the zoom magnification ratio corresponding to thezoom operation is input. In memory 132, the target zoom magnificationratio from the previous process is stored. A multiplier 130 multipliesthe previous target zoom magnification ratio by the update value, and anupdated target zoom magnification ratio is obtained. Note that since theposition of the zoom lens has boundary positions called the wide end andthe tele end, a limiter 132 sets limits so the zoom lens does not exceedthe boundary positions.

The description now returns back to FIG. 6. In step ST103, the targetzoom magnification ratio is stored. If the target zoom magnificationratio was updated, the updated target zoom magnification ratio isstored, whereas if the target zoom magnification ratio was not updated,the previous target zoom magnification ratio is stored. Subsequently,the process proceeds to step ST104.

In step ST104, a smoothing process is conducted on the target zoommagnification ratio, and a smoothed target zoom magnification ratio isobtained. The smoothed target zoom magnification ratio is set as themain image zoom magnification ratio. Subsequently, the process proceedsto step ST105.

In step ST105, zoom control is performed to displace the zoom lens to aposition corresponding to the main image zoom magnification ratio. Thiscontrol is conducted on the lens driving unit 108 from the zoom controlunit 120. According to this control, the lens driving unit 108 moves thezoom lens to a certain position. Subsequently, the process proceeds tostep ST106.

In step ST106, a process of setting an electronic zoom magnificationratio is conducted. The electronic zoom magnification ratio is set avalue according to the difference between the target zoom magnificationratio and the smoothed, and in the direction that cancels out thesmoothing, for example. Subsequently, the process proceeds to stepST107.

In step ST107, zoom control is conducted to set the electronic zoommagnification ratio in the electronic zoom processing unit 116. Theelectronic zoom processing unit 116 electronically zooms the monitoringimage at the set electronic zoom magnification ratio. The monitoringimage that has been optically zoomed according to the main image zoommagnification ratio and electronically zoomed according to theelectronic zoom magnification ratio is displayed on the display device105.

Note that the periods by which the respective processes are conductedare not necessarily the same, and may be different. For example, theperiod of monitoring the presence or absence of the input of a zoomoperation and the period of setting the main image zoom magnificationratio and the electronic zoom magnification ratio may be different.

As described above, in the first embodiment, an image in which thesubject is enlarged/reduced smoothly may be obtained. This image may berecorded to a recording device as appropriate. When playing back therecorded image, since the subject is enlarged/reduced smoothly, it ispossible to prevent the users from feeling a sense of discomfort orunnaturalness. In addition, even if the user is a novice, the user isable to obtain an image in which zoom operations are performed like anexpert. On the other hand, since an image reflecting the user's zoomoperations is displayed during the zoom operations, it is possible toprevent the user from feeling that usability is poor.

2. Second Embodiment

Next, a second embodiment will be described. For the configuration ofthe zoom control device according to the second embodiment, theconfiguration of the zoom control device described in the firstembodiment may be applied, unless specifically indicated otherwise. Thisapplies similarly to the configuration of the imaging device. Likecomponents and processes will be denoted with like reference signs.Also, unless specifically indicated otherwise, the matter described inthe first embodiment is applicable to the second embodiment.

In the first embodiment discussed above, the smoothing parameters thatthe smoothing unit 126 uses in the smoothing process are fixed. In thesecond embodiment, depending on the imaging conditions, the smoothingparameters are changed, and the degree of smoothing is changed. As thedegree of smoothing increases, the magnitude of the effect produced bysmoothing increases. Note that the degree of smoothing may be adjustedby changing the coefficients of the FIR filter, for example. If thecoefficients are assigned to yield a low-pass filter having a low cutofffrequency, the degree of smoothing may be increased, whereas if thecoefficients are assigned to yield a low-pass filter having a highcutoff frequency, the degree of smoothing may be decreased.

The imaging conditions may be, for example, the operational conditionsof the imaging device 1, the conditions of user operations on theimaging device 1, and the imaging mode set in the imaging device 1. Thedegree of smoothing may be, for example, whether the smoothing processis on/off, “large”, in which a strong smoothing process is applied, and“small”, in which a weak smoothing process is applied. Obviously, thesedegrees are merely an example, and the degree of smoothing may also beset to a greater number of levels. The smoothing parameters (forexample, the coefficients of the FIR filter) corresponding to the degreeof smoothing may be set as appropriate.

[Degree of Smoothing According to Imaging Conditions]

FIG. 8 illustrates an example of the degree of smoothing according toimaging conditions. The “recording state” may be given as an example ofthe operational conditions of the imaging device 1. When the “recordingstate” is “recording”, the smoothing process is conducted (on) to raisethe quality of the recorded image. When the “recording state” is“standing by to record”, the smoothing process is not conducted (off) tosimplify processing and improve system response. Note that the “standingby to record” state is a state in which the monitoring image isdisplayed on the display unit, but the main image is not recorded.

The “current position of the zoom lens” may be given as an example ofthe operational conditions of the imaging device 1. When the “currentposition of the zoom lens” is positioned at the end (on either the teleside or the wide side), the smoothing process is not conducted (off) sothat the zoom lens does not move past the end.

The “movement conditions of the imaging device 1” may be given as anexample of the operational conditions of the imaging device 1. The“movement conditions of the imaging device 1” are, for example, themovement speed in the pan direction or the tilt direction, and may beacquired by a sensor such as a gyro sensor. The configuration is notlimited to a physical sensor, and the movement speed of the imagingdevice 1 may also be acquired based on image signal processing. When themovement speed of the imaging device 1 is a fast, the user is consideredto be performing intense camera work intentionally, and thus the degreeof smoothing is set to “small”. The smoothing process may also not beconducted.

The “vibration magnitude of the imaging device 1” may be given as anexample of the conditions of user operations on the imaging device 1.The “vibration magnitude of the imaging device 1” may be acquired byusing a gyro sensor or the like. When the “vibration magnitude of theimaging device 1” is “large”, there is a possibility that the user isperforming intense camera work intentionally, and thus the degree ofsmoothing is set to “small”. The smoothing process may also not beconducted. On the other hand, when the “vibration magnitude of theimaging device 1” is “small”, the degree of smoothing is set to normalor “large” to raise the quality of the recorded image.

A “user indication for a fast zoom operation method” may be given as anexample of the conditions of user operations on the imaging device 1.The “zoom operation method” includes a normal zoom operation method, anda fast zoom operation method that moves the zoom faster compared to thenormal zoom operation method. Which zoom operation method is indicatedby the user may be detected by providing a pressure sensor on the zoomkey, for example. When the fast zoom operation method is indicated bythe user, the degree of smoothing is set to “small” and Vlimit is set toa larger value, so that the speed of the zoom lens becomes the maximumspeed or a speed close to the maximum.

A “normal imaging mode”, an “overcrank imaging mode”, an “undercrankimaging mode”, a “landscape mode”, a “night mode”, a “sports mode”, anda “frame rate setting” may be given as examples of the imaging mode ofthe imaging device 1. “Landscape mode” and the “night mode” are examplesof static image imaging modes that image a static image. “Sports mode”is an example of a dynamic image imaging mode that images a dynamicimage. With the “undercrank imaging mode”, changes in zoom appear fasterthan actual when viewing the image, and thus the degree of smoothing isset to “large” compared to the “normal imaging mode”. On the other hand,with the “overcrank imaging mode”, changes in zoom appear slower thanactual when viewing the image, and thus the degree of smoothing is setto “small” compared to the “normal imaging mode”.

In the case of “landscape mode”, the motion of the imaging device 1 isconsidered to be small. For this reason, the degree of smoothing is setto “large” to raise the quality of the recorded image. In the case of“night mode”, the degree of smoothing is set to “large”, for reasonssimilar to “landscape mode”. In the case of “sports mode”, it isconsidered that the imaging device 1 is subjected to large movementsintentionally. Accordingly, the degree of smoothing is set to “small”.

When the “frame rate” is low, the exposure time is long, and thus blurdue to zooming occurs more readily. Accordingly, the movement speed ofthe zoom lens is limited to a slower speed, and in addition, the degreeof smoothing is set to “large” to increase the amount of smoothing. Inthis way, the degree of smoothing may be varied according to the imagingconditions.

Smoothing parameters corresponding to respective imaging conditions arestored in memory as a table, for example. The zoom control unit 120acquires the smoothing parameters corresponding to the imaging mode byreading the table. The zoom control unit 120 may also be configured tofind the smoothing parameters corresponding to the imaging mode bycomputation. A configuration enabling smoothing parameters to beacquired externally through a network or the like is also acceptable.

[Process Flow]

FIG. 9 is a flowchart illustrating an example of the flow of a processaccording to the second embodiment. Hereinafter, the points that differfrom the process flow of the first embodiment will be describedprimarily.

In the process according to the second embodiment, the processes of stepST201 and step ST202 are added. In step ST201, the imaging conditionsare acquired. The zoom control unit 120 acquires the imaging conditionsfrom input into the operating unit 107 and the like. Subsequently, theprocess proceeds to step ST202.

In step ST202, smoothing parameters are set according to the imagingconditions. For example, the coefficients of the FIR filter are set tosuitable values according to the imaging conditions. Subsequently, theprocess proceeds to step ST104.

In step ST104, a smoothing process is conducted with the smoothingparameters set in step ST202, and the main image zoom magnificationratio is set. Since the other processes are similar to the firstembodiment, duplicate description will be omitted.

As above, according to the second embodiment, a smoothing process may beconducted to a suitable degree according to the imaging conditions. Forthis reason, when the user is shooting with intense camera work, forexample, it is possible to prevent the smoothing process from beingconducted and an image different from the user's intended image frombeing generated.

3. Third Embodiment

Next, a third embodiment will be described. For the configuration of thezoom control device according to the third embodiment, the configurationof the zoom control device described in the first embodiment may beapplied, unless specifically indicated otherwise. This applies similarlyto the configuration of the imaging device. Like components andprocesses will be denoted with like reference signs. Also, unlessspecifically indicated otherwise, the matter described in the first andsecond embodiments is applicable to the third embodiment. In the thirdembodiment, an electronic zoom magnification ratio accounting forprocess delay is set by conducting a synchronization process.

FIG. 10 is a diagram for explaining an example in which an electroniczoom magnification ratio accounting for process delay is set. Theinterval T between the dashed lines is the control interval, and is thescreen update interval, for example.

FIG. 10A illustrates the target zoom magnification ratio correspondingto the user's zoom operations. FIG. 10B illustrates the main image zoommagnification ratio. When obtaining the main image zoom magnificationratio, since a smoothing process is performed on the target zoommagnification ratio and mechanical operations for driving the zoom lensare conducted, a corresponding delay is produced. In FIG. 10B, anillustration accounting for this delay is shown.

FIG. 10C illustrates the zoom magnification ratio of the readout imagefrom the image sensor 102. From the radiation of light onto the imagesensor 102 until the acquisition of image data, a delay corresponding tothe exposure time of the image sensor 102 is produced. In FIG. 10C, anillustration accounting for the delay described in FIG. 10B as well asthe delay corresponding to the exposure time of the image sensor 102 isshown.

FIG. 10D illustrates the zoom magnification ratio of the image inputinto the electronic zoom processing unit 116. At this point, a delaycorresponding to the processing by the digital signal processing unit103 is produced. In FIG. 10, an illustration accounting for the delaydescribed in FIGS. 10B and 10C as well as the delay corresponding to theprocessing by the digital signal processing unit 103 is shown. FIG. 10Eis the magnification ratio of the image displayed on the display device105.

FIGS. 10A to 10E will be used to describe an example of asynchronization process specifically. The zoom key is operated by theuser. Suppose that the target zoom magnification ratio corresponding tothe zoom key operation is a magnification ratio of 2 (×2.0) (FIG. 10A).A smoothing process is conducted on the target zoom magnification ratio,and as the main image zoom magnification ratio, a magnification ratio of1.3 (×1.3) is obtained, for example (FIG. 10B). Subject light taken inat the zoom lens position with an optical zoom magnification ratio of1.3 irradiates and exposes the image sensor 102 (FIG. 10C). Imageprocessing by the digital signal processing unit 103 is conducted on theimage data output from the image sensor 102 (FIG. 10D).

In the case of not accounting for the delay produced by the processes sofar, it is sufficient for the electronic zoom magnification ratio of theelectronic zoom processing unit 116 to approach the target zoommagnification ratio of 2.0. However, in some cases, additional zoomoperations may be performed while a delay is produced, and the targetzoom magnification ratio may change from 2.0. In FIG. 10A, an example isillustrated in which the target zoom magnification ratio changes from2.0 to 4.0 (×4.0). Accordingly, the electronic zoom magnification ratiois set to approach the target zoom magnification ratio of 4.0.Specifically, the electronic zoom magnification ratio is set to 3.07(4.0/1.3).

[Process Flow]

FIG. 11 is a flowchart illustrating an example of the flow of a processaccording to the third embodiment. Hereinafter, the points that differfrom the process flow of the first embodiment will be describedprimarily.

In step ST301, the synchronization process is conducted and theelectronic zoom magnification ratio is set. In step ST107, theelectronic zoom magnification ratio set in step ST301 is set in theelectronic zoom processing unit 116. The electronic zoom processing unit116 conducts electronic zooming according to the set electronic zoommagnification ratio. Since the other processes are similar to theprocesses in the first embodiment, duplicate description will beomitted.

As above, according to the third embodiment, an image that correctlyreflects a zoom magnification ratio corresponding to the user's zoomoperations may be obtained. Furthermore, the user is able to check theimage that correctly reflects the zoom magnification ratio correspondingto the zoom operations.

4. Fourth Embodiment

Next, a fourth embodiment will be described. For the configuration ofthe zoom control device according to the fourth embodiment, theconfiguration of the zoom control device described in the firstembodiment may be applied, unless specifically indicated otherwise. Thisapplies similarly to the configuration of the imaging device. Likecomponents and processes will be denoted with like reference signs.Also, unless specifically indicated otherwise, the matter described inthe first, second, and third embodiments is applicable to the fourthembodiment.

As discussed above, in the first to the third embodiments, the field ofview of the main image and the field of view of the monitor display aremade to be different. Described specifically using FIG. 1, themonitoring image displayed on the display device 105 is an image inwhich the subsequently is zoomed in or zoomed out suddenly to reflectthe user's zoom operations, like from the image M1, to the image M2, tothe image M3. On the other hand, the image recorded to the recordingdevice 104 is an image in which the subject is zoomed in (or zoomed out)smoothly, like from the image M1, to the image M4, to the image M3.Since there is a difference between the displayed image and the recordedimage, there is a risk of the user being unable to grasp the extent ofwhat is being recorded with only the monitoring image. For this reason,there is a possibility that an object not visible on the screen may berecorded, for example. The fourth embodiment is an embodiment devised inlight of this point.

The zoom control unit 120 sets the monitoring image zoom magnificationratio based on Math. 4 below, for example.

Z _(digital) =Z _(target) −Z _(capture)  [Math. 4]

In Math. 4, Ztarget represents the target zoom magnification ratio.Zcapture represents the zoom magnification ratio obtained at the currentzoom lens position (the zoom magnification ratio obtained by opticalzooming), or in other words, the zoom magnification ratio of the imageinput into the electronic zoom processing unit 116. Zdigital representsthe electronic zoom magnification ratio with respect to the monitoringimage. Note each value of Z is expressed in a logarithmicrepresentation. As expressed in Math. 4, Zdigital is set according tothe difference between Ztarget and Zcapture. In the first, second, andthird embodiments discussed earlier, electronic zooming is conducted atthe zoom magnification ratio expressed by Zdigital, regardless of theposition in the screen. In the fourth embodiment, the zoom magnificationratio is changed according to the position in the screen.

For example, provided that P(x_in, y_in) is a certain position of themain image before the electronic zoom process, and P(x_out, y_out) isthe position after the electronic zoom process, the position in thex-axis direction is expressed by Math. 5 below, while the position inthe y-axis direction is expressed by Math. 6 below. Note that Zh inMath. 5 and Zv in Math. 6 represent the magnification ratio in thex-axis direction and the magnification ratio in the y-axis direction ofZdigital discussed above, respectively. The magnification ratio referredto herein is the magnification ratio of the distance from a referencepoint. The reference point may be set arbitrarily, but herein, thereference point is set in the center of the main image. At the referencepoint, x_in=y_in=x_out=y_out=0.

x_in=x_out÷Zh  [Math. 5]

y_in=y_out÷Zv  [Math. 6]

FIG. 12 illustrates an example of the relationship between the positionin the x-axis direction of the main image and the zoom magnificationratio in the case of conducting an enlargement process by electroniczooming. In FIG. 12, the horizontal axis represents the position in thex-axis direction of the main image (center and both edges), while thevertical axis represents the magnification ratio Zh. Note that, althoughnot illustrated, the relationship between the y-axis direction of themain image and the magnification ratio Zv is also similar to FIG. 12. Asillustrated in FIG. 12, if the position in the main image becomes acertain distance or greater from the reference point (for example, thecenter of the image (x_in=0)), the electronic zoom magnification ratiois reduced by Zh, with the electronic zoom magnification ratio being setto correspond to a magnification ratio of 1 at the screen edges.

FIG. 13 illustrates an example of the relationship between the positionin the x-axis direction of the main image (center and both edges) andthe zoom magnification ratio in the case of conducting a reductionprocess by electronic zooming. The horizontal axis represents theposition in the x-axis direction of the main image, while the verticalaxis represents the magnification ratio Zh. Note that, although notillustrated, the relationship between the y-axis direction of the mainimage and the magnification ratio Zv is also similar to FIG. 13. Asillustrated in FIG. 13, if the position in the main image becomes acertain distance or greater from the reference point (for example, thecenter of the image (x_in=0)), the electronic zoom magnification ratiois enlarged by Zh, with the electronic zoom magnification ratio beingset to correspond to a magnification ratio of 1 at the screen edges.

FIG. 14 illustrates a specific example of image changes in the case ofconducting an enlargement process by electronic zooming. The horizontalaxis represents the time axis, while the vertical axis represents thezoom magnification ratio. In FIG. 14, the solid line represents changein the main image zoom magnification ratio, while the dashed linerepresents the monitoring image zoom magnification ratio. Details aboutthe image M10, the image M20, the image M20′ and image M30, and theimage M40 in FIG. 14 will be discussed later. Note that the image M10,the image M20, the image M20′ and image M30, and the image M40 areillustrated enlarged for the sake of convenience.

From time t0 to time t1, an initial image, namely the image M10, isdisplayed on the display unit. The image M10 includes three persons MA1near the center, a person MA2 near the edge on the left side, a personMA3 near the edge on the right side, and the sun SU near the edge ontop.

At time t1, a zoom operation (in this example, a zoom-in operation) isperformed by the user. The image enlarged by the target zoommagnification ratio obtained according to the zoom operation is theimage M20′. Also, the image obtained by the main image zoommagnification ratio between time t2 and time t3 is the main image M40.If the image M20′ is displayed as the monitoring image at time t2, adifference from the image M40 is produced. Specifically, although theperson MA2 and the person MA3 are not displayed in the image M20′, theperson MA2 and the person MA3 are included in the main image M40. Forthis reason, when the main image M40 is recorded, an image including theperson MA2 and the person MA3 not being displayed to the user isrecorded, and there is a risk of recording an image different from theuser's intended image.

Accordingly, as described using FIG. 12, the electronic zoommagnification ratio near the center of the image is set equal to thetarget zoom magnification ratio, with the electronic zoom magnificationratio becoming smaller with increasing distance away from the area nearthe center. At this point, making the electronic zoom magnificationratio near the center of the image equal to the target zoommagnification ratio is because typically the subject that the user wantsto zoom is often positioned in the center, and the user is able torecognize how this subject actually is enlarged in response to zoomoperations.

The output image obtained in the case of conducting electronic zoomingby the electronic zoom magnification ratio illustrated as an example inFIG. 12 is the image M20. When the image M20 is compared to the imageM20′ obtained by the target zoom magnification ratio, the imagemagnification ratios near the center of the images are equal.Consequently, the user is able to check how the subject near the centeris enlarged in response to zoom operations. Additionally, at theperiphery of the image M20, although the size of the image is reducedand the aspect ratio has changed, the person MA2, the person MA3, andthe sun SU are displayed. Consequently, the user is able to checkobjects and the like displayed at the periphery, and recognize thatthese objects and the like will be recorded. Note that when zoomingfinally is completed, the main image zoom magnification ratio and thetarget zoom magnification ratio become equal, and the distortion at theperiphery is resolved.

According to the fourth embodiment, since the display in the center ofthe monitoring image becomes an image reflecting the target zoommagnification ratio, the user is able to perform zoom operations withoutbeing aware of the smoothing process. Furthermore, since the peripheryof the monitoring image indicates the recording area, for example, theuser is able to check whether or not unwanted subjects will be depictedin the recorded image.

5. Modifications

The foregoing thus specifically describes embodiments of the presentdisclosure, but the present disclosure is not limited to the foregoingembodiments, and various modifications based on the technical ideas ofthe present disclosure are possible. Hereinafter, exemplarymodifications will be described.

FIG. 15 illustrates an example of the configuration of an imaging deviceaccording to a modification. In FIG. 15, part of the configuration ofthe imaging device is illustrated, while the configuration of theimaging device 1 discussed earlier may be applied for the part of theconfiguration that is not illustrated. As illustrated in FIG. 15, thedigital signal processing unit 103 is provided with an electronic zoomprocessing unit 210. Additionally, a main image zoom magnification ratiocontrol unit 220 and a monitoring image zoom magnification ratio controlunit 230 are provided. The main image zoom magnification ratio controlunit 220 controls the electronic zoom processing unit 210, while themonitoring image zoom magnification ratio control unit 230 controls theelectronic zoom processing unit 210 and the electronic zoom processingunit 116 in a compound manner. Zoom control similar to the opticalzooming on the main image in the foregoing embodiments may also beconducted through electronic zooming by the electronic zoom processingunit 210. Zoom control with respect to the monitoring image is realizedby compound control through a first electronic zoom control by theelectronic zoom processing unit 210 and a second electronic zoom controlby the electronic zoom processing unit 116.

FIG. 16 illustrates an example of the configuration of an imaging deviceaccording to another modification. In FIG. 16, part of the configurationof the imaging device is illustrated, while the configuration of theimaging device 1 discussed earlier may be applied for the part of theconfiguration that is not illustrated. The imaging device illustrated inFIG. 16 includes a main image zoom magnification ratio control unit 240and a monitoring image zoom magnification ratio control unit. The mainimage zoom magnification ratio control unit 240 controls the electroniczoom processing unit 210, while the monitoring image zoom magnificationratio control unit 250 controls the electronic zoom processing unit 116.As illustrated in FIG. 16, when conducting a zoom-out process, the imagefrom before electronic zooming is conducted by the electronic zoomprocessing unit 210 may be input into the electronic zoom processingunit 116. In this case, it is sufficient for the electronic zoomprocessing unit 210 to conduct the zoom control indicated by the solidline in FIG. 1, and the electronic zoom processing unit 116 to conductthe zoom control indicated by the dashed line in FIG. 1, for example. Inthis way, the zoom control with respect to the main image and the zoomcontrol with respect to the monitoring image may also be electronic zoomcontrols that are parallel to each other.

In the case of taking the configuration illustrated in FIG. 16, thedriving of the zoom lens with respect to the lens block 101 may also beconducted manually. In this case, the optical zoom magnification ratiobecomes the target zoom magnification ratio. Accordingly, the image tobe recorded is made to reach the main image zoom magnification ratio bythe processing performed by the electronic zoom processing unit 210.Note that in this case, since the magnification ratio of the electroniczoom processing unit 116 used in the monitoring image is always 1, whenlens driving is conducted only manually, a configuration that omits theelectronic zoom processing unit 116 is also acceptable.

The present disclosure is not limited to being a device, and may also berealized by a method, a program, a system, or the like. A program may beprovided to the user over a network, or through portable memory such asan optical disc or semiconductor memory.

Note that the configurations and processes in the embodiments and themodifications may also be combined appropriately, insofar as a technicalcontradiction is not produced. The order of the respective processes inthe process flows given as an example may also be modifiedappropriately, insofar as a technical contradiction is not produced.

The present disclosure may also be applied to what is called a cloudsystem, in which the processes given as an example are processed in adistributed manner by multiple devices. The present disclosure may alsobe realized in a system in which the processes given as an example inthe embodiments and the modifications are executed, as a device in whichat least some of the processes given as an example are executed.

Additionally, the present technology may also be configured as below.

(1)

A zoom control device, including:

-   -   a zoom magnification ratio change speed setting unit that sets a        main image zoom magnification ratio change speed and a        monitoring image zoom magnification ratio change speed according        to a zoom operation by a user; and    -   a zoom control unit that conducts a zoom control on a main image        so that a zoom magnification ratio changes according to the main        image zoom magnification ratio change speed, and conducts a zoom        control on a monitoring image so that the zoom magnification        ratio changes according to the monitoring image zoom        magnification ratio change speed, wherein    -   the zoom magnification ratio change speed setting unit is        configured to set the main image zoom magnification ratio change        speed by smoothing the monitoring image zoom magnification ratio        change speed.        (2)

The zoom control device according to (1), wherein

-   -   the zoom control on the main image is an optical zoom control,        and    -   the zoom control on the monitoring image is a compound control        made up of the optical zoom control, and an electronic zoom        control on an image obtained according to the optical zoom        control.        (3)

The zoom control device according to (2), wherein

-   -   the zoom magnification ratio change speed setting unit        -   sets, as the main image zoom magnification ratio change            speed, a smoothed target zoom magnification ratio change            speed obtained by smoothing a target zoom magnification            ratio change speed obtained according to the zoom operation,            and        -   sets the monitoring image zoom magnification ratio change            speed to a value combining the main image zoom magnification            ratio change speed and an electronic zoom magnification            ratio change speed decided based on the target zoom            magnification ratio change speed and the smoothed target            zoom magnification ratio change speed.            (4)

The zoom control device according to (3), wherein

-   -   the zoom magnification ratio change speed setting unit sets the        electronic zoom magnification ratio change speed to a value in a        direction that cancels out the smoothing.        (5)

The zoom control device according to (4), wherein

-   -   the zoom magnification ratio change speed setting unit changes a        degree of smoothing according to an imaging condition.        (6)

The zoom control device according to (5), wherein

-   -   the imaging condition includes recording and standing by to        record, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing between the recording and the standing        by to record.        (7)

The zoom control device according to (5), wherein

-   -   the imaging condition includes a current position of a zoom        lens, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing according to the current position of the        zoom lens.        (8)

The zoom control device according to (5), wherein

-   -   the imaging condition includes movement conditions of an imaging        device, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing according to the movement conditions of        the imaging device.        (9)

The zoom control device according to (5), wherein

-   -   the imaging condition includes a presence or absence of a user        indication for a zoom operation method which moves zoom faster        compared to a normal zoom operation method and which is        different from the normal zoom operation method, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing according to the presence or absence of        the user indication.        (10)

The zoom control device according to (5), wherein

-   -   the imaging condition includes an imaging mode, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing according to the imaging mode.        (11)

The zoom control device according to (10), wherein

-   -   the imaging mode includes at least one of an overcrank imaging        mode and an undercrank imaging mode, and a normal imaging mode,        and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing between the at least one of the        overcrank imaging mode and the undercrank imaging mode, and the        normal imaging mode.        (12)

The zoom control device according to (10), wherein

-   -   the imaging mode includes a static image imaging mode that        images a static image, and a dynamic image imaging mode that        images a dynamic image, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing between the static image imaging mode        and the dynamic image imaging mode.        (13)

The zoom control device according to (5), wherein

-   -   the imaging condition includes a frame rate, and    -   the zoom magnification ratio change speed setting unit changes        the degree of smoothing according to the frame rate.        (14)

The zoom control device according to any of (1) to (13), wherein

-   -   the zoom magnification ratio change speed setting unit sets the        monitoring image zoom magnification ratio change speed to        different speeds depending on a position in a screen.        (15)

The zoom control device according to (14), wherein

-   -   the zoom magnification ratio change speed setting unit sets the        monitoring image zoom magnification ratio change speed to        mutually different speeds between an area near a center and an        area near an edge in the screen.        (16)

The zoom control device according to (1), wherein

-   -   the zoom control on the main image is a first electronic zoom        control, and    -   the zoom control on the monitoring image is a compound control        made up of the first electronic zoom control, and a second        electronic zoom control on an image that has been subjected to        the first electronic zoom control.        (17)

The zoom control device according to (1), wherein

-   -   the zoom control on the main image and the zoom control on the        monitoring image are electronic zoom controls that are parallel        to each other.        (18)

A zoom control method, including:

-   -   setting, by a zoom magnification ratio change speed setting        unit, a main image zoom magnification ratio change speed and a        monitoring image zoom magnification ratio change speed according        to a zoom operation by a user; and    -   conducting, by a zoom control unit, a zoom control on a main        image so that a zoom magnification ratio changes according to        the main image zoom magnification ratio change speed, and        conducting a zoom control on a monitoring image so that the zoom        magnification ratio changes according to the monitoring image        zoom magnification ratio change speed, wherein    -   the zoom magnification ratio change speed setting unit sets the        main image zoom magnification ratio change speed by smoothing        the monitoring image zoom magnification ratio change speed.        (19)

A program causing a computer to execute a zoom control method including:

-   -   setting, by a zoom magnification ratio change speed setting        unit, a main image zoom magnification ratio change speed and a        monitoring image zoom magnification ratio change speed according        to a zoom operation by a user; and    -   conducting, by a zoom control unit, a zoom control on a main        image so that a zoom magnification ratio changes according to        the main image zoom magnification ratio change speed, and        conducting a zoom control on a monitoring image so that the zoom        magnification ratio changes according to the monitoring image        zoom magnification ratio change speed, wherein    -   the zoom magnification ratio change speed setting unit sets the        main image zoom magnification ratio change speed by smoothing        the monitoring image zoom magnification ratio change speed.        (20)

A zoom control device, including:

-   -   a monitoring image zoom magnification ratio control unit that        controls an operation of changing a monitoring image zoom        magnification ratio according to a monitoring image zoom        magnification ratio change speed based on a zoom operation by a        user; and    -   a main image zoom magnification ratio control unit that controls        an operation of changing a main image zoom magnification ratio        according to a main image zoom magnification ratio change speed        obtained by conducting a smoothing process on the monitoring        image zoom magnification ratio change speed.        (21)

An imaging device, including:

-   -   a zoom magnification ratio change speed setting unit that sets a        main image zoom magnification ratio change speed and a        monitoring image zoom magnification ratio change speed according        to a zoom operation by a user; and    -   a zoom control unit that conducts a zoom control on a main image        so that a zoom magnification ratio changes according to the main        image zoom magnification ratio change speed, and conducts a zoom        control on a monitoring image so that the zoom magnification        ratio changes according to the monitoring image zoom        magnification ratio change speed, wherein    -   the zoom magnification ratio change speed setting unit is        configured to set the main image zoom magnification ratio change        speed by smoothing the monitoring image zoom magnification ratio        change speed.        (22)

An imaging device, including:

-   -   a monitoring image zoom magnification ratio control unit that        controls an operation of changing a monitoring image zoom        magnification ratio according to a monitoring image zoom        magnification ratio change speed based on a zoom operation by a        user; and    -   a main image zoom magnification ratio control unit that controls        an operation of changing a main image zoom magnification ratio        according to a main image zoom magnification ratio change speed        obtained by conducting a smoothing process on the monitoring        image zoom magnification ratio change speed.

REFERENCE SIGNS LIST

-   1 imaging device-   104 recording device-   105 display device-   106 control unit-   107 operating unit-   108 lens driving unit-   116 electronic zoom processing unit-   120 zoom control unit

1. A zoom control device, comprising: a zoom magnification ratio changespeed setting unit that sets a main image zoom magnification ratiochange speed and a monitoring image zoom magnification ratio changespeed according to a zoom operation by a user; and a zoom control unitthat conducts a zoom control on a main image so that a zoommagnification ratio changes according to the main image zoommagnification ratio change speed, and conducts a zoom control on amonitoring image so that the zoom magnification ratio changes accordingto the monitoring image zoom magnification ratio change speed, whereinthe zoom magnification ratio change speed setting unit is configured toset the main image zoom magnification ratio change speed by smoothingthe monitoring image zoom magnification ratio change speed.
 2. The zoomcontrol device according to claim 1, wherein the zoom control on themain image is an optical zoom control, and the zoom control on themonitoring image is a compound control made up of the optical zoomcontrol, and an electronic zoom control on an image obtained accordingto the optical zoom control.
 3. The zoom control device according toclaim 2, wherein the zoom magnification ratio change speed setting unitsets, as the main image zoom magnification ratio change speed, asmoothed target zoom magnification ratio change speed obtained bysmoothing a target zoom magnification ratio change speed obtainedaccording to the zoom operation, and sets the monitoring image zoommagnification ratio change speed to a value combining the main imagezoom magnification ratio change speed and an electronic zoommagnification ratio change speed decided based on the target zoommagnification ratio change speed and the smoothed target zoommagnification ratio change speed.
 4. The zoom control device accordingto claim 3, wherein the zoom magnification ratio change speed settingunit sets the electronic zoom magnification ratio change speed to avalue in a direction that cancels out the smoothing.
 5. The zoom controldevice according to claim 4, wherein the zoom magnification ratio changespeed setting unit changes a degree of smoothing according to an imagingcondition.
 6. The zoom control device according to claim 5, wherein theimaging condition includes recording and standing by to record, and thezoom magnification ratio change speed setting unit changes the degree ofsmoothing between the recording and the standing by to record.
 7. Thezoom control device according to claim 5, wherein the imaging conditionincludes a current position of a zoom lens, and the zoom magnificationratio change speed setting unit changes the degree of smoothingaccording to the current position of the zoom lens.
 8. The zoom controldevice according to claim 5, wherein the imaging condition includesmovement conditions of an imaging device, and the zoom magnificationratio change speed setting unit changes the degree of smoothingaccording to the movement conditions of the imaging device.
 9. The zoomcontrol device according to claim 5, wherein the imaging conditionincludes a presence or absence of a user indication for a zoom operationmethod which moves zoom faster compared to a normal zoom operationmethod and which is different from the normal zoom operation method, andthe zoom magnification ratio change speed setting unit changes thedegree of smoothing according to the presence or absence of the userindication.
 10. The zoom control device according to claim 5, whereinthe imaging condition includes an imaging mode, and the zoommagnification ratio change speed setting unit changes the degree ofsmoothing according to the imaging mode.
 11. The zoom control deviceaccording to claim 10, wherein the imaging mode includes at least one ofan overcrank imaging mode and an undercrank imaging mode, and a normalimaging mode, and the zoom magnification ratio change speed setting unitchanges the degree of smoothing between the at least one of theovercrank imaging mode and the undercrank imaging mode, and the normalimaging mode.
 12. The zoom control device according to claim 10, whereinthe imaging mode includes a static image imaging mode that images astatic image, and a dynamic image imaging mode that images a dynamicimage, and the zoom magnification ratio change speed setting unitchanges the degree of smoothing between the static image imaging modeand the dynamic image imaging mode.
 13. The zoom control deviceaccording to claim 5, wherein the imaging condition includes a framerate, and the zoom magnification ratio change speed setting unit changesthe degree of smoothing according to the frame rate.
 14. The zoomcontrol device according to claim 1, wherein the zoom magnificationratio change speed setting unit sets the monitoring image zoommagnification ratio change speed to different speeds depending on an aposition in a screen.
 15. The zoom control device according to claim 14,wherein the zoom magnification ratio change speed setting unit sets themonitoring image zoom magnification ratio change speed to mutuallydifferent speeds between an area near a center and an area near an edgein the screen.
 16. The zoom control device according to claim 1, whereinthe zoom control on the main image is a first electronic zoom control,and the zoom control on the monitoring image is a compound control madeup of the first electronic zoom control, and a second electronic zoomcontrol on an image that has been subjected to the first electronic zoomcontrol.
 17. The zoom control device according to claim 1, wherein thezoom control on the main image and the zoom control on the monitoringimage are electronic zoom controls that are parallel to each other. 18.A zoom control method, comprising: setting, by a zoom magnificationratio change speed setting unit, a main image zoom magnification ratiochange speed and a monitoring image zoom magnification ratio changespeed according to a zoom operation by a user; and conducting, by a zoomcontrol unit, a zoom control on a main image so that a zoommagnification ratio changes according to the main image zoommagnification ratio change speed, and conducting a zoom control on amonitoring image so that the zoom magnification ratio changes accordingto the monitoring image zoom magnification ratio change speed, whereinthe zoom magnification ratio change speed setting unit sets the mainimage zoom magnification ratio change speed by smoothing the monitoringimage zoom magnification ratio change speed.
 19. A program causing acomputer to execute a zoom control method comprising: setting, by a zoommagnification ratio change speed setting unit, a main image zoommagnification ratio change speed and a monitoring image zoommagnification ratio change speed according to a zoom operation by auser; and conducting, by a zoom control unit, a zoom control on a mainimage so that a zoom magnification ratio changes according to the mainimage zoom magnification ratio change speed, and conducting a zoomcontrol on a monitoring image so that the zoom magnification ratiochanges according to the monitoring image zoom magnification ratiochange speed, wherein the zoom magnification ratio change speed settingunit sets the main image zoom magnification ratio change speed bysmoothing the monitoring image zoom magnification ratio change speed.20. A zoom control device, comprising: a monitoring image zoommagnification ratio control unit that controls an operation of changinga monitoring image zoom magnification ratio according to a monitoringimage zoom magnification ratio change speed based on a zoom operation bya user; and a main image zoom magnification ratio control unit thatcontrols an operation of changing a main image zoom magnification ratioaccording to a main image zoom magnification ratio change speed obtainedby conducting a smoothing process on the monitoring image zoommagnification ratio change speed.